The present disclosure relates to augmented reality and more particularly to a stand-alone augmented reality camera that is utilized to generate two-dimensional (2D) images that are formed into three-dimensional (3D) images through use of scale measurements made by 3D metrology equipment.
Augmented reality (AR) is a relatively new type of technology that grew out of virtual reality. Augmented reality merges, superimposes, or transprojects actual real-world information or data with, on, into, or onto virtual information or data. That is, the virtual information or data “augments,” compliments or supplements the actual sensed, measured, captured or imaged real-world information or data related to some object or scene to give the user an enhanced view or perception of the real world object or scene. Augmented reality applications include technical or industrial areas such as part, component or device manufacturing and assembly and/or repair and maintenance, and facility, building or structure layout and construction. A number of modern-day AR applications are disclosed at http://en.wikipedia.org/wild/Augmented_reality.
The actual information or data relating to the part, component or device, or area or scene, may be obtained in various ways using various devices. One type of device that may provide the actual information or data includes a 3D metrology device, such as, for example, a coordinate measurement device in the nature of a portable articulated arm coordinate measurement machine (AACMM) or a laser tracker. This type of measurement device may measure and provide the actual 3D coordinates of the part, component, device, area or scene in the nature of the three translational coordinates (e.g., the x, y and z or Cartesian coordinates) as well as the three rotational coordinates (e.g., pitch, roll and yaw). As such, the measurement device may be understood as providing for six degrees of freedom (i.e., six-DOF).
A camera may also be used to take still or video images of the actual part, component or device, and/or a desired area or scene by itself or that surrounding or associated with the part, component or device.
The virtual information or data may be stored artificial information regarding the part, component, device, area or scene. The stored virtual information or data may be related to the design of the part, component, device, area or scene ranging from, for example, simple text or symbols to relatively more complex, graphic 3D CAD design data. Besides visual information, the stored virtual information or data may also comprise audible or sound information or data. The stored virtual information or data may also relate to information such as textual or part, component or device repair or maintenance instructions, or visual information depicting parts, components or devices that may be used, for example, in the design of an office or manufacturing and/or repair facility (e.g., a building or facility layout).
The combined actual and virtual information or data in an AR system is usually digital in nature and may be delivered in real-time (i.e., as the actual information is being measured or sensed) to a user on a display screen that may be in many different types or forms, such as that associated with, for example, a desktop or laptop computer monitor, tablet, smartphone or even a head-mounted display such as those associated with glasses, hats or helmets. Audio information may be delivered through a speaker.
As mentioned, one type of 3D metrology or coordinate measurement device includes a portable AACMM. These AACMMs have found widespread use in the manufacturing or production of parts where there is a need to rapidly and accurately verify the dimensions of the part during various stages of the manufacturing or production (e.g., machining). Portable AACMMs represent an improvement over known stationary or fixed, cost-intensive and relatively difficult to use measurement installations, particularly in the amount of time it takes to perform dimensional measurements of relatively complex parts. Typically, a user of a portable AACMM simply guides a probe along the surface of the part or object to be measured. The measurement data are then recorded and provided to the user. In some cases, the data are provided to the user in visual form, for example, in 3D form on a computer screen. In other cases, the data are provided to the user in numeric form, for example when measuring the diameter of a hole, the text “Diameter=1.0034” is displayed on a computer screen.
An example of a prior art portable AACMM is disclosed in U.S. Pat. No. 5,402,582 ('582) to Raab, the contents of which are incorporated herein by reference. The '582 patent discloses a 3D measurement system comprised of a manually-operated AACMM having a support base on one end and a measurement probe at the other end. Also, U.S. Pat. No. 5,611,147 ('147) to Raab, the contents of which are incorporated herein by reference, discloses a similar AACMM. In the '147 patent, the AACMM includes a number of features including an additional rotational axis at the probe end, thereby providing for an arm with either a two-two-two or a two-two-three axis configuration (the latter case being a seven axis arm).
Another type of 3D metrology or coordinate measurement device belongs to a class of instruments known as a laser tracker that measures the 3D coordinates of a point by sending a laser beam to the point. The laser beam may impinge directly on the point or on a retroreflector target in contact with the point. In either case, the laser tracker instrument determines the coordinates of the point by measuring the distance and the two angles to the target. The distance is measured with a distance measuring device such as an absolute distance meter or an interferometer. The angles are measured with an angle measuring device such as an angular encoder. A gimbaled beam-steering mechanism within the instrument directs the laser beam to the point of interest.
The laser tracker is a particular type of coordinate measurement device that tracks the retroreflector target with one or more laser beams it emits. The laser tracker is thus a “time-of-flight” (TOF) type of measurement device. Coordinate measurement devices closely related to the laser tracker are the laser scanner and the total station. The laser scanner steps one or more laser beams to points on a surface of an object. It picks up light scattered from the surface and from this light determines the distance and two angles to each point. The total station, which is most often used in surveying applications, may be used to measure the coordinates of diffusely scattering or retroreflective targets.
Ordinarily the laser tracker sends a laser beam to a retroreflector target. A common type of retroreflector target is the spherically mounted retroreflector (SMR), which comprises a cube-corner retroreflector embedded within a metal sphere. The cube-corner retroreflector comprises three mutually perpendicular mirrors. The vertex, which is the common point of intersection of the three mirrors, is located at the center of the sphere. Because of this placement of the cube corner within the sphere, the perpendicular distance from the vertex to any surface on which the SMR rests remains constant, even as the SMR is rotated. Consequently, the laser tracker can measure the 3D coordinates of the object surface by following the position of an SMR as it is moved over the surface. Stating this another way, the laser tracker needs to measure only three degrees of freedom (one radial distance and two angles) to fully characterize the 3D coordinates of a surface.
One type of laser tracker contains only an interferometer (IFM) without an absolute distance meter (ADM). If an object blocks the path of the laser beam from one of these trackers, the IFM loses its distance reference. The operator must then track the retroreflector to a known location to reset to a reference distance before continuing the measurement. A way around this limitation is to put an ADM in the tracker. The ADM can measure distance in a point-and-shoot manner, as described in more detail below. Some laser trackers contain only an ADM without an interferometer. U.S. Pat. No. 7,352,446 ('446) to Bridges et al., the contents of which are incorporated herein by reference, describes a laser tracker having only an ADM (and no IFM) that is able to accurately scan a moving target. Prior to the '446 patent, absolute distance meters were too slow to accurately find the position of a moving target.
A gimbal mechanism within the laser tracker may be used to direct a laser beam from the tracker to the SMR. Part of the light retroreflected by the SMR enters the laser tracker and passes onto a position detector. A control system within the laser tracker can use the position of the light on the position detector to adjust the rotation angles of the mechanical axes of the laser tracker to keep the laser beam centered on the SMR. In this way, the tracker is able to follow (track) an SMR that is moved over the surface of an object of interest.
Angle measuring devices such as angular encoders are attached to the mechanical axes of the tracker. The one distance measurement and two angle measurements performed by the laser tracker are sufficient to completely specify the three-dimensional location of the SMR at any point on the surface of the object being measured.
Several laser trackers have been disclosed for measuring six, rather than the ordinary three, degrees of freedom. These six degrees of freedom include three translational degrees of freedom and three orientational degrees of freedom, as described in more detail hereinafter. Exemplary six degree-of-freedom (six-DOF or 6-DOF) laser tracker systems are described by U.S. Pat. No. 7,800,758 ('758) to Bridges et al., U.S. Pat. No. 8,525,983 ('983) to Bridges et al., and U.S. Pat. No. 8,467,072 ('072) to Cramer et al., the contents of each of which are incorporated herein by reference.
These six-DOF laser trackers may include a separate probe having a retroreflector for which the laser tracker measures the six degrees of freedom. The six degrees of freedom of the probe measured by the laser tracker may be considered to include three translational degrees of freedom and three orientational degrees of freedom. The three translational degrees of freedom may include a radial distance measurement between the laser tracker and the retroreflector, a first angular measurement, and a second angular measurement. The radial distance measurement may be made with an IFM or an ADM within the laser tracker. The first angular measurement may be made with an azimuth angular measurement device, such as an azimuth angular encoder, and the second angular measurement made with a zenith angular measurement device, such as a zenith angular encoder. Alternatively, the first angular measurement device may be the zenith angular measurement device and the second angular measurement device may be the azimuth angular measurement device. The radial distance, first angular measurement, and second angular measurement constitute three coordinates in a spherical coordinate system, which can be transformed into three coordinates in a Cartesian coordinate system or another coordinate system.
The three orientational degrees of freedom of the probe may be determined using a patterned cube corner, as described in the aforementioned patent '758. Alternatively, other methods of determining the three orientational degrees of freedom of the probe may be used. The three translational degrees of freedom and the three orientational degrees of freedom fully define the position and orientation of the six-DOF probe (and, thus, of the probe tip) in space. It is important to note that this is the case for the systems considered here because it is possible to have systems in which the six degrees of freedom are not independent so that six degrees of freedom are not sufficient to fully define the position and orientation of a device in space. The term “translational set” is a shorthand notation for three degrees of translational freedom of a six-DOF accessory (such as the six-DOF probe) in the laser tracker frame of reference. The term “orientational set” is a shorthand notation for three orientational degrees of freedom of a six-DOF accessory (e.g., the probe) in the laser tracker frame of reference. The term “surface set” is a shorthand notation for three-dimensional coordinates of a point on the object surface in the laser tracker frame of reference as measured by the probe tip.
Other known types of 3D metrology devices include the aforementioned TOF laser scanners and also triangulation scanners.
While some innovations have already been made in the area of augmented reality for use with various types of 3D metrology devices, there is a need for novel applications of augmented reality for use with 3D metrology devices such as AACMMs, laser trackers, TOF laser scanners, and triangulation scanners.